Generating Reference Texts for Short Answer Scoring Using Graph-based Summarization

Automated scoring of short answers often involves matching a students response against one or more sample reference texts. Each reference text provided contains very specific instances of correct responses and may not cover the variety of possibly correct responses. Finding or hand-creating additional references can be very time consuming and expensive. In order to overcome this problem we propose a technique to generate alternative reference texts by summarizing the content of top-scoring student responses. We use a graph-based cohesion technique that extracts the most representative answers from among the top-scorers. We also use a state-of-the-art extractive summarization tool called MEAD. The extracted set of responses may be used as alternative reference texts to score student responses. We evaluate this approach on short answer data from Semeval 2013’s Joint Student Response Analysis task.